Understanding Synonymous Referring Expressions via Contrastive Features

نویسندگان

چکیده

Abstract Referring expression comprehension aims to localize objects identified by natural language descriptions. This is a challenging task as it requires understanding of both visual and domains. One nature that each object can be described synonymous sentences with paraphrases, such varieties in languages have critical impact on learning model. While prior work usually treats sentence attends an separately, we focus referring model considers the property sentences. To this end, develop end-to-end trainable framework learn contrastive features image instance levels, where extracted from describe same should closer other after mapping domain. We conduct extensive experiments evaluate proposed algorithm several benchmark datasets, demonstrate our method performs favorably against state-of-the-art approaches. Furthermore, since expressions become larger across datasets when they different ways, present cross-dataset transfer settings validate ability learned transferable features.

برای دانلود باید عضویت طلایی داشته باشید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Generating Contrastive Referring Expressions

The referring expressions (REs) produced by a natural language generation (NLG) system can be misunderstood by the hearer, even when they are semantically correct. In an interactive setting, the NLG system can try to recognize such misunderstandings and correct them. We present an algorithm for generating corrective REs that use contrastive focus (“no, the BLUE button”) to emphasize the informa...

متن کامل

Understanding referring expressions in a person-machine spoken dialogue

In the domain of mobile robotic task execution under dialogue control, a primary goal is to identify the task target which is speci ed by a natural language description. A number of concepts are expressed in the user spoken language by vague terms like \the big box" and \very close to the door". We use fuzzy logic to map these vague terms onto the quantitative data collected by system sensors. ...

متن کامل

Planning English Referring Expressions

This paper describes a theory of language generation based on planning. To illustrate the theory, the problem of planning referring expressions is examined in detail. A theory based on planning makes it possible for one to account for noun phrases that refer, that inform the hearer of additional information, and that are coordinated with the speaker's physical actions to clarify his communicati...

متن کامل

Iterations in Referring Expressions

In this paper, research is done to examine the usefulness of ’iterations’ in generated referring expressions (that is, using multiple sentences to form a referring expression (with and/or without feedback from the hearer) as an expression instead of one long sentence, without any pause or breaks). Two experiments have been done, one to examine how humans use referring expressions and the second...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: International Journal of Computer Vision

سال: 2022

ISSN: ['0920-5691', '1573-1405']

DOI: https://doi.org/10.1007/s11263-022-01647-z